As part of a Fundamentals of HCI course, I worked on a project that involved the usability evaluation of a website. I performed a usability test with five users, as well as a heuristic evaluation of the website.
I chose to evaluate the University of Maryland transportation interface. The reason I picked this website was that I believed there were several gaps in the users’ understanding of the services and information offered by the website. My peers and I, despite requiring the information provided on the website, barely use the website due to the lack of ease of obtaining said information. I wanted to gain a sense of what these gaps specifically were and how they could be improved.
TARGET USER
A typical user for the interface would be students, faculty, or staff at the University of Maryland. Since the website provides information to a group of people that goes beyond just the students at the university, the age range could be described as anyone over the age of 18.
The website is fairly straightforward in terms of its features, a user would, therefore, not require immense computational skills further than using a computer, or a handheld mobile device and being familiar with using the internet.
The website is fairly straightforward in terms of its features, a user would, therefore, not require immense computational skills further than using a computer, or a handheld mobile device and being familiar with using the internet.
TASKLIST
The task list was created keeping in mind the several services offered by the website. It encompasses all the different modes of transport, therefore giving the user a chance to explore every facet of the website.
The tasks also varied in simplicity, to test out the basic flow and presentation of information as well as the more complex features provided on the interface, such as the interactive maps, and the integration of an application within the interface.
1. Find out which shuttle route ends at Greenbelt. - the goal for this task was to evaluate how easily the users found the different bus routes available people at the University of Maryland.
2. Find out which of the routes are functional in the evenings only. - the goal for this task was to evaluate where users were able to find this information, since it was possible to seek it out from multiple avenues and whether the tag marking evening buses on the schedule page was enough of an indicator.
3. Find out how many bike repair stations are available by the Kim Engineering Building. - the goal for this task was to understand whether users were able to find the map with the information easily and whether the map gave them all the required data.
4. Find out how many minutes later will the shuttle to College Park Metro arrive at the Mitchell Building. - the goal for this task was to test out the usability of the Nextbus application integration on the website and whether users felt comfortable enough to use it at all.
5. Find out the three different types of Parking Permits available to students. - the goal for this task was to understand how easily users were able to find the information considering the vast number of pages and test out the clarity of the way the information was categorised under the Parking tab of the website.
6. Find out how many drivers are available to carpool with from 32 Greenhill Road, Greenbelt, to the Art Sociology Building on weekdays. (Do not sign up or log in to the website to perform this task) - the goal for this task was to test out a form on the website that users need to fill out to acquire results..
7. You decide to travel for the summer. Find out the storage fee to park your e-scooter in a secure lot on campus for two months. - the goal for this task was to understand how easy or difficult categorisation and naming of the pages under the bikes and eScooters section made it to seek data out of.
8. Find out where the Bike Locks and Lights sale is being held on the 30th of November? - the goal for this task was to gain an understanding of which of the two available paths to the information is preferred and picked more intuitively by the users. (the information was available both on the homepage and the events page of the website.)
9. Your car tires require inflation. Find out whether the Motorist Assistance Vehicle will be able to assist you with inflating your tires. - the goal for this task was to evaluate how easily users were able to seek out the relevant from a long dropdown of pages. Additionally the page with the information about the Motorist Assistance vehicle was placed under the Parking tab on the website. I wanted to gain an understanding of whether this placement hindered users from completing the task.
10. Find out how many electric vehicle charging stations are available at Union Lane Garage. - the goal for this task was to understand whether users were able to find the map with the information easily and whether the map gave them all the required data.
Users were selected amongst friends from different academic and professional backgrounds. All the users are students at the University of Maryland, therefore making them potential users of the website I selected. However, this was the first time each user was actually using the UMD transportation website. All the users, being graduate students, were fairly competent in their technical skills. The tests were completed at the users’ convenience between the 18th - 20th of November.
U1 - Female; graduate student; 25-40; computer engineer; technical expert
U2 - Female; graduate student; 25-40; business analyst; technically competent
U3 - Female; graduate student; 18-25; marketing analyst; technically competent
U4 - Male; graduate student; 18-25; business analyst; technically competent
U5 - Female; graduate student; 18-25; business analyst; technically competent
USER TASK AND TIME PERFORMANCE
Task 1, Finding a shuttle route - Average time - 2:12; Success rate - ⅘
Three of the five users started this task by going to the ‘New user guide’ page under the Shuttle-UM tab in the header menu.
U2, who was unable to successfully find the correct answer to the question went to one of the route maps provided, and they were unable to find the right answer due to a lack of all the bus routes being covered and clearly marked on the map.
U2, who was unable to successfully find the correct answer to the question went to one of the route maps provided, and they were unable to find the right answer due to a lack of all the bus routes being covered and clearly marked on the map.
Task 2, Shuttle working hours - Average time - 00:39; Success rate - ⅘
This task was completed successfully by all the users except U4. The task was fairly straightforward once the user was on the shuttle schedule page. However, finding the information while on another page might prove confusing, which is why U4 went to the ‘nite ride’ page under the shuttle-um tab to look for the evening shuttle services.
Task 3, Bike repair stations - Average time - 02:10; Success rate - ⅗
For this task, all the users were able to get to the correct page on the website with the map marking out the bike racks and the repair stations. However, the map provided no visual guide to determine which marked spots are racks and which ones are repair stations. The only place this information is provided is on the previous page, where the user is unable to see the map. Due to this, U2, and U4 found the map with the correct information but were unable to correctly identify which marked areas were the repair stations.
Task 4, Shuttle tracking - Average time - 2:42; Success rate - ⅕
This task was the one with the least success rate. The website uses an integration of the ‘Nextbus’ application into the interface to track the schedules. Despite being avid users of the Nextbus app, none of the users, other than U5, were able to find and use the feature.
Task 5, Parking permits - Average time - 01:07; Success rate - 5/5
This was one of the few tasks that all the users were able to complete, thanks to the clear distribution of data within the parking tab on the menu. However, all of the users scrolled by the first link on the page that led to the different permit types. They only noticed the link after scrolling back up to the top of the page. This, therefore, increased the average time the users took to complete the task and speaks to the hierarchy of the information on the website and poses the question of whether the important links are highlighted enough.
Task 6, Carpooling - Average time - 02:16; Success rate - ⅗
This task was fairly straightforward in terms of finding the right page and entering the data. However, once the results were shown on the page, I noticed that the page yielded results of not only carpooling options but also other transit options despite it being a page dedicated to carpools. This confused some of the users and they scrolled further down the page to make sure they hadn’t missed any other important information.
Additionally, the results of the carpool search sometimes involve repetitions of what seems like the same people and the number of options always shows up as exactly 20 despite there not being 20 options. This is highly confusing to a user using the system for the first time.
Additionally, the results of the carpool search sometimes involve repetitions of what seems like the same people and the number of options always shows up as exactly 20 despite there not being 20 options. This is highly confusing to a user using the system for the first time.
Task 7, eScooter storage fee - Average time - 03:11; Success rate - ⅖
This task had a low success rate due to the misplacement of information. The storage fee for both bikes as well as e-scooters was under the bike resources page of the website despite having a page dedicated to e-scooters. This, therefore, caused most users to first go to the e-scooters page looking for the relevant information.
Task 8, Lights and locks sale - Average time - 01:31; Success rate - ⅘
All the users except U2 were able to complete this task. However, they were also able to eventually find the right answer on the homepage of the website. Out of the five users, two of them found the information on the homepage of the website. While the other three went to the events page under the about section of the website, they seemed unsure of whether they were going to the right page and probably did not expect events to be under the about section of the website.
Task 9, MAV services - Average time - 01:11; Success rate - 5/5
All the users were able to complete this task successfully within a short period. However, they did struggle to find the right page immediately due to the large number of pages categorised in the header menu of the website. Additionally so because unlike Shuttle-UM, and Bikes and eScooters, the website menu lacks a dropdown specifically for motorised vehicles. Information about motorised assistance vehicles is, therefore, placed under the parking dropdown in the top menu of the website. This leaves room for some confusion.
Task 10, EV charging stations - Average time - 02:09; Success rate - 5/5
All the users were able to quickly find the map marking out the EV charging stations on campus, since there was a very clearly named menu item under the sustainable transport dropdown, telling them where they would find the information. Additionally, on the page, the link to the map is clearly marked right below the header image and is given enough visual importance such that there is a very slim chance that the user might miss the link and scroll past it.
ANALYSIS
One of the major problems I noticed due to the results of the user based testing is the way information is organised and marked on the website. Users often struggled with finding the right pages that would hold the necessary details and often ended up on the wrong pages before they were able to find the right one, if at all.
For example, for the first task, three out of the five users went to the ‘new rider guide’ page to find out what the different shuttle routes were. On the website, however, the new rider guide page is purely a set of instructions related to social conduct and mask mandates on the busses. For task 7, all of the users first visited the eScooters page to find the storage fee of an eScooter on campus. The parking fee is only mentioned on the Bike Resources page and not on the eScooters page, despite it being relevant to both. This, therefore, increased the average time users spent on this task.
For example, for the first task, three out of the five users went to the ‘new rider guide’ page to find out what the different shuttle routes were. On the website, however, the new rider guide page is purely a set of instructions related to social conduct and mask mandates on the busses. For task 7, all of the users first visited the eScooters page to find the storage fee of an eScooter on campus. The parking fee is only mentioned on the Bike Resources page and not on the eScooters page, despite it being relevant to both. This, therefore, increased the average time users spent on this task.
Apart from the organisation of the content, one major drawback I found was the lack of visibility of the search icon on the website. Only two out of the five users used the search function. Altering the placement of the search icon, perhaps beside the header menu on the website would give it more visibility and would, thus, make it a more useful feature to users.
